DCU at VideoClef 2008

We describe a baseline system for the VideoCLEF Vid2RSS task. The system uses an unaltered off-the-shelf Information Retrieval system. ASR content is indexed using default stemming and stopping methods. The subject categories are populated by using the category label as a query on the collection, and assigning the retrieved items to that particular category. We describe the results of the system and provide some high-level analysis of its performance

Overview of VideoCLEF 2008: Automatic generation of topic-based feeds for dual language audio-visual content

The VideoCLEF track, introduced in 2008, aims to develop and evaluate tasks related to analysis of and access to multilingual multimedia content. In its first year, VideoCLEF piloted the Vid2RSS task, whose main subtask was the classification of dual language video (Dutchlanguage television content featuring English-speaking experts and studio guests). The task offered two additional discretionary subtasks: feed translation and automatic keyframe extraction. Task participants were supplied with Dutch archival metadata, Dutch speech transcripts, English speech transcripts and 10 thematic category labels, which they were required to assign to the test set videos. The videos were grouped by class label into topic-based RSS-feeds, displaying title, description and keyframe for each video. Five groups participated in the 2008 VideoCLEF track. Participants were required to collect their own training data; both Wikipedia and general web content were used. Groups deployed various classifiers (SVM, Naive Bayes and k-NN) or treated the problem as an information retrieval task. Both the Dutch speech transcripts and the archival metadata performed well as sources of indexing features, but no group succeeded in exploiting combinations of feature sources to significantly enhance performance. A small scale fluency/adequacy evaluation of the translation task output revealed the translation to be of sufficient quality to make it valuable to a non-Dutch speaking English speaker. For keyframe extraction, the strategy chosen was to select the keyframe from the shot with the most representative speech transcript content. The automatically selected shots were shown, with a small user study, to be competitive with manually selected shots. Future years of VideoCLEF will aim to expand the corpus and the class label list, as well as to extend the track to additional tasks

Classification of dual language audio-visual content: Introduction to the VideoCLEF 2008 pilot benchmark evaluation task

VideoCLEF is a new track for the CLEF 2008 campaign. This track aims to develop and evaluate tasks in analyzing multilingual video content. A pilot of a Vid2RSS task involving assigning thematic class labels to video kicks off the VideoCLEF track in 2008. Task participants deliver classification results in the form of a series of feeds, one for each thematic class. The data for the task are dual language television documentaries. Dutch is the dominant language and English-language content (mostly interviews) is embedded. Participants are provided with speech recognition transcripts of the data in both Dutch and English, and also with metadata generated by archivists. In addition to the classification task, participants can choose to participate in a translation task (translating the feed into a language of their choice) and a keyframe selection task (choosing a semantically appropriate keyframe for depiction of the videos in the feed)

Multimedia retrieval in MultiMatch: The impact of speech transcript errors on search behaviour

This study discusses the findings of an evaluation study on the performance of a multimedia multimodal information access sub-system (MIAS), incorporating automatic speech recognition technology (ASR) to automatically transcribe the speech content of video soundtracks. The study’s results indicate that an information-rich but minimalist graphical interface is preferred. It was also discovered that users tend to have a misplaced confidence in the accuracy of ASR-generated speech transcripts, thus they are not inclined to conduct a systematic auditory inspection (their usual search behaviour) of a video’s soundtrack if the query term does not appear in the transcript. In order to alert the user to the possibility that a search term may be incorrectly recognised as some other word, a matching algorithm is proposed that searches for word sequences of similar phonemic structure to the query term

Examining the contributions of automatic speech transcriptions and metadata sources for searching spontaneous conversational speech

The searching spontaneous speech can be enhanced by combining automatic speech transcriptions with semantically related metadata. An important question is what can be expected from search of such transcriptions and different sources of related metadata in terms of retrieval effectiveness. The Cross-Language Speech Retrieval (CL-SR) track at recent CLEF workshops provides a spontaneous speech test collection with manual and automatically derived metadata fields. Using this collection we investigate the comparative search effectiveness of individual fields comprising automated transcriptions and the available metadata. A further important question is how transcriptions and metadata should be combined for the greatest benefit to search accuracy. We compare simple field merging of individual fields with the extended BM25 model for weighted field combination (BM25F). Results indicate that BM25F can produce improved search accuracy, but that it is currently important to set its parameters suitably using a suitable training set

Multilingual search for cultural heritage archives via combining multiple translation resources

The linguistic features of material in Cultural Heritage (CH) archives may be in various languages requiring a facility for effective multilingual search. The specialised language often associated with CH content introduces problems for automatic translation to support search applications. The MultiMatch project is focused on enabling users to interact with CH content across different media types and languages. We present results from a MultiMatch study exploring various translation techniques for the CH domain. Our experiments examine translation techniques for the English language CLEF 2006 Cross-Language Speech Retrieval (CL-SR) task using Spanish, French and German queries. Results compare effectiveness of our query translation against a monolingual baseline and show improvement when combining a domain-specific translation lexicon with a standard machine translation system

Domain-specific query translation for multilingual access to digital libraries

Accurate high-coverage translation is a vital component of reliable cross language information access (CLIR) systems. This is particularly true of access to archives such as Digital Libraries which are often specific to certain domains. While general machine translation (MT) has been shown to be effective for CLIR tasks in information retrieval evaluation workshops, it is not well suited to specialized tasks where domain specific translations are required. We demonstrate that effective query translation in the domain of cultural heritage (CH) can be achieved by augmenting a standard MT system with domain-specific phrase dictionaries automatically mined from the online Wikipedia. Experiments using our hybrid translation system with sample query logs from users of CH websites demonstrate a large improvement in the accuracy of domain specific phrase detection and translation

Periodicity detection in lifelog data with missing and irregularly sampled data

Lifelogging is the ambient, continuous digital recording of a person’s everyday activities for a variety of possible applications. Much of the work to date in lifelogging has focused on developing sensors, capturing information, processing it into events and then supporting event-based access to the lifelog for applications like memory recall, behaviour analysis or similar. With the recent arrival of aggregating platforms such as Apple’s HealthKit, Microsoft’s HealthVault and Google’s Fit, we are now able to collect and aggregate data from lifelog sensors, to centralize the management of data and in particular to search for and detect patterns of usage for individuals and across populations. In this paper, we present a framework that detects both lowlevel and high-level periodicity in lifelog data, detecting hidden patterns of which users would not otherwise be aware. We detect periodicities of time series using a combination of correlograms and periodograms, using various signal processing algorithms. Periodicity detection in lifelogs is particularly challenging because the lifelog data itself is not always continuous and can have gaps as users may use their lifelog devices intermittingly. To illustrate that periodicity can be detected from such data, we apply periodicity detection on three lifelog datasets with varying levels of completeness and accuracy

Dementia ambient care: multi-sensor support to enable independent home-based living for people with dementia

The quality of life of the person with dementia is often impacted by the loss of autonomy and independence that can arise during dementia progression. Ambient assistive technologies represent a way of enabling independence and facilitating “ageing in place”, by supporting the health, lifestyle, and safety of the person with dementia in an unobtrusive manner. Dem@Care is a European FP7 funded project, which is using ambient and wearable sensors to maintain independent home-based living for as long as possible. We have identified five frequently problematic areas for the person with dementia that can be supported by technology: Sleep, Activities of Daily Living, Physical Activity, Social Interaction, and Mood. In Dem@Care, a clinical assessment is carried out with the person with dementia and their family to identify their unique needs in each of the 5 areas. An individualised sensor “toolbox” is tailored and discussed with the individual and their family, and an acceptable and useful system is configured and deployed. Over time, information gathered by sensors is used to provide feedback to identify changes in patterns of behaviour that may indicate deterioration, improvement, stasis, or the risk of future deterioration, and to increase awareness of behaviours that are detrimental to health and well-being. We report relevant guiding principles from the literature, and findings from the first Dem@Care pilot evaluation, regarding user-centred design, individualization, ethics, and the acceptability and usability of current Dem@Care sensors. We present results from the monitoring of sleep, physical activity, and daily-living activities and following promising initial results, we are expanding data collection to incorporate additional sensors and new participants with the expectation that we can demonstrate the ability of the Dem@Care system to enable persons with dementia to remain independent and living in their own homes for longer

Pattern detection in lifelog data

Lifelogging technology is getting attention from industry, academic and market, such as wearable sensors and the concept of smart home. We are proposing a framework which could handle those aggregated multimodal and longitudinal data. The system will take advantage of the rich information carried chronologically and implement process such as data cleaning, low and high level patterns detection and giving feedback to users